Method and apparatus for controlling information in virtual world

ABSTRACT

Provided is a method for controlling information shared in a virtual world by a server and a plurality of users&#39; client terminals connected to each other through a network. In this information control method, a computer creates a third object between a first object and a second object in the virtual world when the first object and second object transmit communication information therebetween, and filters the information with the third object. In addition, when an object transmits visual information, the third object can be created around the object transmitting the visual information. The third object can be created as a pseudo wall.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a technique of controlling an elementconstituting a virtual world, in the virtual world.

2. Description of the Related Art

In a virtual world, a three-dimensional virtual space created by aserver on a network is shared by multiple users' client terminals. Inthis virtual world, a character (hereinafter, referred to as an avatar)representing a user can move freely in the space, and can enjoy thevirtual world, as if the virtual world were an extension in the realworld, by communicating with the avatars of other users and by shopping.The virtual world is especially characterized in such a way that a largenumber of users from a wide variety of various places join the virtualworld that a user can communicate with various unknown users in realtime.

While allowing a user to communicate with an unspecified number ofusers, the virtual world is desired to provide an opportunity tocommunicate only among specified users. Japanese Patent ApplicationPublication No. 2002-215553 describes a technique of constructing andcontrolling a space for a meeting, a lecture or the like, suited tocommunicate with specific users.

To be more precise, a moderator establishes a space, and has anauthority to control operations executed by browsers of all users in thespace. Types of such space include (1) a class room type, (2) a roundtable type, (3) a self-schooling type and (4) a compound type of a classroom type and a round table type. In any type of space, all the avatarsexcept for the moderator are not allowed to exchange information withanother avatar outside the space. In (2) the round table type,participating users share information and are allowed to exchangeinformation with each other through their avatars. In addition, theauthority given to each avatar limits a range of the reach of a messagefrom the avatar and a range of browsers controllable by the avatar. Asdescribed above, the conventional technique enables the access controlfor avatars existing inside the space by constructing and managing thespace.

Accordingly, there are several problems with establishing andcontrolling a space in a virtual world. One problem is collision(interference) between objects belonging to a space and objects such asavatars existing inside the virtual world. As a result, the space itselfand the objects are so restricted that they cannot operate or actflexibly. Moreover, it is difficult to allow only a remote avatar toenter the space by expanding the space, since an unnecessary avatarexisting on the way to the remote avatar also inevitably enters thespace. Another problem is that controllable accesses are limited. Forexample, controllable accesses can be limited only to accesses acrossthe boundary of the space.

The aforementioned technique enables control only for accesses acrossthe boundary of the space, but the access control for each object isneeded in the virtual world. This is because a wide variety of objectsexist in the virtual world and have different desires. Furthermore, in avirtual world, it is necessary to perform not only access control butalso controls for communication information, visual information andinformation on object movements.

SUMMARY OF THE INVENTION

In consideration of the foregoing, the present invention provides amethod and an apparatus for controlling information shared in a virtualworld by a server and multiple users' client terminals connected througha network.

In one aspect, the present invention provides a method for controllinginformation shared in a virtual world by a server and a plurality ofusers' client terminals connected to each other through a network. Inthis information control method, a computer creates a third objectbetween a first object and a second object in the virtual world when thefirst object and second object transmit communication informationtherebetween, and filters the information with the third object. Inaddition, when an object transmits visual information, the third objectcan be created around the object transmitting the visual information.

Here, an “object” denotes an object, including an avatar, which existsin the virtual world. The “communication information” denotesinformation exchanged between a first object and a second object,specifically, between avatars or between an object and an avatar tocommunicate with each other. A speech and chat are specific examples ofthe “communication information.” The chat indicates the communicationinformation in text. When information is exchanged, speeches sometimesoverlap with each other, while chats never overlap with each other. Thispoint is a difference between the speech and chat.

The “visual information” denotes information on the appearance of anobject in the virtual world. Information on the appearance andbrightness of an object is a specific example of the “visualinformation.” In this description, a simple expression of “information”indicates both the communication information and the visual information.The “filtering” is to change information or to permit/prohibit passing.The filtering is performed by a third object created between a firstobject and a second object. The third object can be a pseudo wall, forexample.

The third object is able to perform different kinds of filteringaccording to moving directions of information and another object.Moreover, the third object is able to permit another object to passthrough the third object, and to prohibit still another object frompassing therethrough.

In addition, another aspect of the present invention can be provided asa computer program to be executed on a computer, or a system having thecomputer program installed thereon.

According to the present invention, the third object that filtersinformation is independent of an object in the virtual world. Thirdobjects in the virtual world are not subject to the constraints of otherobjects that perform filtering. In other words, the third objects arefree from the constraints due to a collision (interference), a distanceand the like between the objects. Different kinds of filtering can beprovided to the respective objects by creating the third object betweenobjects that transmit communication information or around an object thattransmits visual information.

Since the third object allows other objects to pass therethrough, theobjects in the virtual world are not only free from the constraints ofthe object that performs the filtering, but they are also able to behaveas if the third object were not present.

Furthermore, by setting the information to have a direction, the kind offiltering can be changed according to the direction of informationtransmission. In other words, a single third object is made capable ofperforming two or more different kinds of filtering according to thedirections. This enables provision of services satisfying variousdemands of objects.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present invention and theadvantage thereof, reference is now made to the following descriptiontaken in conjunction with the accompanying drawings:

FIG. 1 shows a mechanism of filtering by a pseudo wall object.

FIG. 2 shows an example of a network environment employable in anembodiment of the present invention.

FIG. 3 shows examples of system configuration diagrams of a clientterminal and a server according to the embodiment of the presentinvention.

FIG. 4 shows a processing flow at a time when update information isinputted to the client terminal.

FIG. 5 shows a processing flow in a pseudo wall object creating unit.

FIG. 6 shows a processing flow in a speech processing unit.

FIG. 7 shows a filter processing flow in a chat processing unit.

FIG. 8 shows a processing flow in a filtering unit.

FIG. 9 shows a movement determination processing flow in an objectprocessing unit.

FIG. 10 shows a rendering processing flow in a light processing unit.

FIG. 11 shows a screen image in which an automatic translation pseudowall object is used.

FIG. 12 shows a screen image in which an avatar selection pseudo wallobject is used.

FIG. 13 shows a screen image in which an advertisement change pseudowall object is used.

FIG. 14 shows a screen image in which a pseudo wall object having a dayand night converter function is used.

FIG. 15 shows a hardware configuration of the server or the clientterminal according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, an embodiment of the present invention will be described byreferring to the accompanying drawings. Here, “communicationinformation” is described by taking, as specific examples, speech andchat, and “visual information” is described by taking, as a specificexample, light representing the appearance and brightness of an objectin the virtual world. The virtual world is composed of communicationinformation, visual information and objects, i.e., more precisely,speech, chat, light and objects. These components are called virtualworld components all together, and the components related to information(speech, chat and light) are particularly called virtual world componentinformation.

FIG. 1 shows a mechanism of filtering by a pseudo wall object. Thepseudo wall object is an object in a wall shape partitioning a spacebetween two objects. In FIG. 1, a pseudo wall object 1 exists between anavatar 3 and an avatar 4. A dotted-line arrow 6 indicates virtual worldcomponent information transmitted from the avatar 3 to the avatar 4, anda dotted-line arrow 7 indicates virtual world component informationtransmitted from the avatar 4 to the avatar 3. As an example, thevirtual world component information mentioned here is speech. Thevirtual world component information 6 transmitted from the avatar 3 andthe virtual world component information 7 transmitted from the avatar 4are filtered by the pseudo wall object 1 and then reach the avatar 4 andthe avatar 3, respectively. In other words, the virtual world componentinformation 6 from the avatar 3 to the pseudo wall object 1 is differentfrom the virtual world component information 6 from the pseudo wallobject 1 to the avatar 4, since the virtual world component information6 is filtered by the pseudo wall object 1. Similarly, the virtual worldcomponent information 7 from the avatar 4 to the pseudo wall object 1 isdifferent from the virtual world component information 7 from the pseudowall object 1 to the avatar 3. Specific filtering functions by thepseudo wall object will be described later.

Here, in FIG. 1, one of the surfaces of the pseudo wall object isdefined as a front side, and the other surface is defined as a backside. Since the pseudo wall object has the front and back sides, thepseudo wall object performs two kinds of filtering, one of which is forthe case where the virtual world component information passes from theback side to the front side, and the other of which is for the casewhere the virtual world component information passes from the front sideto the back side. Hereinafter, a forward direction 5 denotes a directionin which the virtual world component information passes from the frontside to the back side, and a reverse direction denotes a direction inwhich the virtual world component information passes from the back sideto the front side. The pseudo wall object 1 can differently filter thevirtual world component information 6 and the virtual world componentinformation 7. Thus, FIG. 1 shows that the pseudo wall object 1 has twofilters, one of which is used for the virtual world componentinformation passing from the front side, and the other of which is usedfor the virtual world component information passing from the back side.Moreover, an effective range can be set for the pseudo wall object. Theeffective range is a range covered by a function of the pseudo wallobject. Here, in FIG. 1, assume that the pseudo wall object 1 iseffective only for the avatar 3 and the avatar 4. In this case, theeffective range is expressed as an effective range 2, and the pseudowall object 1 has no effect on an avatar existing outside the effectiverange 2. The effective range either can be set by a user who isoperating an avatar or it can be automatically determined by a distance.

Note that an independent object is not limited to the pseudo wall objectin a wall shape, but can be, for example, a pseudo block object in ablock shape, a pseudo curved surface object in a curved surface shape orthe like. Since the pseudo wall object has a wall shape, a space can becreated by combining pseudo wall objects with each other. Thus, such aspace can also be used to filter the virtual world component informationinside and outside the space.

In addition, a pseudo wall object is not always created between avatarsas shown in FIG. 1, but can be created between an avatar and a materialobject, or between material objects. Here, the material object means asubstance existing in the virtual world, and specially, a building, apiece of furniture, an automobile, an animal and the like.

The foregoing description has been for the mechanism of the pseudo wallobject to filter the virtual world component information, instead of amechanism of the pseudo wall object to filter virtual world componentsincluding objects. In the case of filtering an object, not informationbut the object itself, the object is filtered by the pseudo wall object.

In this regard, the two kinds of filtering are different from eachother. Except for this point, however, there is no difference betweenthe mechanism of filtering the virtual world component information, andthe mechanism of filtering the object by means of the pseudo wallobject. Accordingly, the following description will be provided withoutdiscriminating between the virtual world component information and theobject.

A network environment employable in an embodiment of the presentinvention will be described. FIG. 2 shows one example of the networkenvironment employable in the embodiment of the present invention. Thevirtual world is constructed by an information control system 400configured to a server 100 and multiple client terminals (here, twoclient terminals 200 and 201 are illustrated) connected to a network300. By operating the client terminal 200, a user can have a chat or thelike with a user of the client terminal 201 through the network 300 andthe server 100 in the virtual world. Note that two or more servers canbe employed as the server 100 (called a server, below), and three ormore client terminals can be employed as the client terminals 200 and201 (collectively called a client terminal, below).

FIG. 3 shows detailed configurations of the client terminal and theserver. FIG. 3 shows one example of a system configuration diagram ofthe client terminal and the server according to the embodiment of thepresent invention. Hereinafter, a database is simply referred to as aDB. The server includes a pseudo wall object DB 110, a speech DB 111, anavatar DB 112, a chat DB 113, a 3D model DB 114, a pseudo wall objectcreating unit 120, a speech processing unit 121, a chat processing unit122, a filtering unit 123 and a server information update unit 126. Onthe other hand, the client terminal includes a pseudo wall object DB210, an avatar DB 212, a 3D model DB 214, an object processing unit 224,a light processing unit 225, a client information update unit 226, aninput unit 227 and an output unit 228. Although both of the server andthe client terminal have the respective pseudo wall object DBs, avatarDBs and 3D model DBs, the server and the client terminal are differentin that the server has the information on all the avatars while theclient terminal has the information only on an avatar of the user of theclient terminal.

The pseudo wall object creating unit 120 is one example of a thirdobject creating unit, and the server information update unit 126 is oneexample of an information update unit. The client terminal also includesthe same information update unit, and transmits and receives informationto and from the server. The pseudo wall object DBs 110 and 210 areexamples of a third object storage unit. The speech DB 111 and the chatDB 113 are examples of a communication information storage unit. Theavatar DBs 112 and 212 are examples of a visual information storageunit. The 3D model DBs 114 and 214 are examples of a movementinformation storage unit.

In this embodiment, a filtering processing unit includes the speechprocessing unit 121, the chat processing unit 122, the filtering unit123, the object processing unit 224 and the light processing unit 225.While the server includes the filtering unit 123, the client terminalhas no such filtering unit. This is because the filtering processing inthe client terminal is performed by the object processing unit 224 andthe light processing unit 225. In this way, the filtering unit can beprovided inside each processing unit or be provided independently.Moreover, each storage unit is not limited to a DB provided on a harddisk, but can be an external storage device such as a memory, a magnetictape or a flexible disk. Hardware configurations of the server and theclient terminal will be described in the last part of thisspecification.

FIG. 4 shows a processing flow for a case where update information isinputted to the client terminal. When the update information is inputtedto the client terminal through the input unit 227 such as a keyboard,the client information update unit 226 determines whether or not theclient terminal performs processing for the inputted information (S40).If the virtual world component specified by the inputted information isa light or an object, the client terminal performs the processing forthe inputted information. If the client information update unit 226determines that the client terminal is to perform the processing, theinformation is transmitted to the object processing unit 224 or thelight processing unit 225, and then the processing result is reflectedin the avatar DB 212 and the 3D model DB 214 (S41). Regardless ofwhether the client terminal performs the processing, the updateinformation is transmitted from the client information update unit 226to the server information update unit 126 in the server (S42).

Next, the server determines whether or not to perform processing (S43).If the virtual world component indicated by the update information is aspeech or a chat, the server performs the processing for the inputtedinformation. If the server chooses to perform the processing, theinformation is transmitted to the speech processing unit 121 or the chatprocessing unit 122, and one of the corresponding DBs in the server isupdated as a result of the processing (S44). Specifically, if theinputted information is update information of the speech, the speech DB111 is updated. Meanwhile, if the inputted information is updateinformation of the chat, the chat DB 113 is updated. Even though theserver does not perform the processing for the inputted information, theavatar DB 112 and the 3D model DB 114 in the server are updated ifnecessary (S45). This update is for causing the update information ofthe client terminal to be reflected in another client terminal. Theupdate results of the DBs in the server are transmitted from the serverinformation update unit 126 to the client information update unit 226(S46), and the results are outputted through a screen, a speaker and thelike of the output unit 228. Note that if the pseudo wall objectperforms the filtering, the processing units in both the client terminaland the server perform the filtering processing using the respectivepseudo wall object DBs 110 and 210. In the server, particularly, thefiltering unit 123 is called to perform the processing by the speechprocessing unit 121 and the chat processing unit 122.

Hereinafter, descriptions will be given for a function and minimumnecessary data items for each of the DBs. Each of a pair of the avatarDBs 112 and 212, a pair of the 3D model DBs 114 and 214 and a pair ofthe pseudo wall object DBs 110 and 210 respectively included in theserver and the client terminal has the same function and data items.Accordingly, the function and data items will be explained collectivelyfor each pair of the two DBs.

Speech DB 111

This is a DB for storing, for each piece of speech data, an avatarmaking speech, an addressed avatar and the piece of speech data. TheSpeech DB 111 stores the following data items:

-   -   An avatar ID generating speech,    -   An addressed avatar ID, and    -   Speech data.        Avatar DB 112 and 212

This is a DB for storing, for each avatar, the avatar ID, and the 3Dmodel of the avatar. The avatar DB stores the following data items:

-   -   An avatar ID, and    -   A 3D model.        Chat DB 113

This is a DB for storing, for each piece of chat data, an avatartransmitting the piece of chat data, an addressed avatar and the pieceof chat data. The Chat DB 113 stores the following data items:

-   -   An avatar ID transmitting chat data,    -   An addressed avatar ID, and    -   Chat data.        3D Model DBs 114 and 214

This is a DB for storing the ID and the 3D model of each material orbackground object. The 3D model DB stores the following data items:

-   -   A material or background object ID, and    -   A 3D model        Pseudo Wall Object DBs 110 and 210

This is a DB for storing the information on each pseudo wall object. Thepseudo wall object DB stores the following data items:

-   -   A pseudo wall object ID    -   A 3D model    -   A forward direction (normal) definition    -   A speech filter (forward direction)    -   A speech filter (reverse direction)    -   A chat filter (forward direction)    -   A chat filter (reverse direction)    -   A light filter (forward direction)    -   A light filter (reverse direction)    -   An object filter (forward direction)    -   An object filter (reverse direction)

The shapes of objects are set in 3D models in the 3D model DBs 114 and214, and in the pseudo wall object DBs 110 and 210. The pseudo wallobject DBs 110 and 210 each have information on a wall shape and a walltransparency such as a thin wall like a curtain and a transparent walllike a glass. A wall invisible to the avatars can also be set.

Here, the filter information indicates a combination of a filteringfunction corresponding to each moving direction of a virtual worldcomponent and information related to the function. In this embodiment,the filter information for each of the pseudo wall object DB s 110 and210 includes the forward direction (normal) definition, the speechfilter (forward direction), the speech filter (reverse direction), thechat filter (forward direction), the chat filter (reverse direction),the light filter (forward direction), the light filter (reversedirection), the object filter (forward direction), and the object filter(reverse direction). The filtering function of a filter for virtualworld components, such as the speech filter, can be roughly categorizedinto two types: pass/no-pass and transformation functions. Each or bothof the two filtering functions are defined for each of the filters. Thepass/no-pass function is to allow or not to allow a virtual worldcomponent to pass through a pseudo wall object. The transformationfunction is to transform a virtual world component into a differentcomponent when the component passes through the pseudo wall object. Inparticular, the pseudo wall object allowing an object to passtherethrough is called non-solid, while the pseudo wall objectprohibiting an object from passing therethrough is called solid.Hereinafter, whether or not an object is allowed to pass is indicated bywhether a pseudo wall object is non-solid or solid; this attribute iscalled a solid attribute. In addition, an item of the solid attributecan be independently defined in the pseudo wall object DBs 110 and 210in order to simply define the solid attribute of a pseudo wall objectregardless of a moving direction of an object, where the object ismainly an avatar.

The forward direction (normal) definition includes information fordefining the front surface of a pseudo wall object and the informationis used to determine in which direction of the forward and reversedirections a virtual world component passes through the pseudo wallobject. The filtering function is defined not only for each virtualworld component but also for each of moving directions of each virtualworld component in the virtual world. A direction of a virtual worldcomponent is determined according to the forward direction (normal)definition. In addition, two processing functions of each filter areseparately stored for the respective moving directions of the virtualworld component, i.e., the forward direction and reverse direction, whenthe component passes through the filter. The function of the filter inthe reverse direction for the virtual world component is not necessarilythe reverse of that in the forward direction. Precisely, even if thefunction of a certain filter in the forward direction is to translate avirtual world component from English to Japanese, the function of thefilter in the reverse direction is not limited to the function ofreversely translating a virtual world component from Japanese toEnglish.

The pseudo wall object can function as a speech filter for a virtualworld component in the forward direction and can function as a lightfilter for a virtual world component in the reverse direction. Moreover,a pseudo wall object can function as a filter for several kinds ofvirtual world components passing in one of the directions. Thus, thefunction of a single pseudo wall object is not necessarily limited tothe one for only a single kind of virtual world component. This isapparent from the fact that the pseudo wall object DBs 110 and 210 havethe processing function items for several kinds of virtual worldcomponents with respect to a single pseudo wall object.

In addition, in order to process a single kind of virtual worldcomponent through multiple different kinds of filtering, a single pseudowall object can have multiple filtering functions, or, alternatively,multiple pseudo wall objects having different functions can besuperposed on one another. An example of multiple pseudo wall objectssuperimposed on one another is the implementation of a function ofconverting speech in Japanese at a normal speed into the speech inEnglish at a slow speed. This function is implemented by superimposing apseudo wall object of translating Japanese into English on a pseudo wallobject of changing a speaking speed. In this way, the superimposing ofpseudo wall objects enables various conversions.

The avatar can freely create and delete a pseudo wall object. Forinstance, the user of an avatar can create a pseudo wall object having anecessary function when the avatar makes conversions with another avatarand he or she can delete the pseudo wall object when the conversationsare finished. Otherwise, a computer can automatically create a pseudowall object as circumstances demand. For example, a computer can captureinformation on an avatar approaching a particular advertisement and canautomatically create a pseudo wall object according to the capturedinformation to show the content suitable for the avatar.

In order to filter the speech or chat, a pseudo wall object is createdbetween objects having a verbal conversation or chat. Meanwhile, inorder to perform the light filtering, a pseudo wall object can becreated around an object for which the light filtering is desired. Suchan object has an appearance that is different from another object orviews another object differently. Such creation of a pseudo wall objectsurrounding an object is needed for the light filtering, because thespeech filtering and chat filtering are targeted for particular objectswhile the light filtering and object filtering are targeted for a largenumber of unspecified objects in some cases.

FIG. 5 shows a processing flow in the pseudo wall object creating unit120. The pseudo wall object creating unit 120 starts the processing uponreceipt of parameters for a pseudo wall object when a command to createthe pseudo wall object is issued in response to an instruction by a useror in response to a determination by a computer. The pseudo wall objectcreating unit 120 obtains the information on the authority level fromthe avatar DB 112 (S50) and checks the authority level (S51). Ifauthority is given, the pseudo wall object creating unit 120 checks theconfiguration parameters of the pseudo wall object (S52) and determineswhether or not the parameters include an invalid parameter (S53). If noinvalid parameter is included, the pseudo wall object creating unit 120creates a pseudo wall object, stores the information on the createdobject in the pseudo wall object DB 110 (S54), and then terminates theprocessing. In contrast, if authority is not given, or if an invalidparameter is included, the pseudo wall object creating unit 120terminates the processing without creating any pseudo wall object. Thepseudo wall object thus created has a filtering function for a virtualworld component.

Hereinafter, descriptions will be provided for the filtering function ofa pseudo wall object for each kind of virtual world components. Thespecific examples will be described below as Examples 1 to 6.

Speech Filtering

An embodiment of the present invention includes the pseudo wall objecthaving the speech filter function. A speech made by an avatar isconverted when passing through the pseudo wall object having the speechfilter, and thereby another avatar listens to the converted speech. Asdescribed above, the speech is processed by the speech processing unit121 in the server.

FIG. 6 shows a filtering processing flow in the speech processing unit121. When the user of an avatar ‘a’ inputs a speech through the inputunit 227 such as a microphone, the speech inputted to the speech DB 111is stored through the information update units 226 and 126 respectivelyin the client terminal and in the server (S60). Thereafter, from thepseudo wall object DB 110, the speech processing unit 121 obtains theinformation on a pseudo wall object existing within the reach of thespeech from the avatar ‘a’ (S61). A range of the reach of the speech canbe set by the user of the avatar ‘a’, or can be automatically set. Therange is specified by use of the avatar IDs, the distance from theavatar making the speech, or the like. The speech processing unit 121determines whether or not the information on the pseudo wall object isobtained in S61 (S62). If the information is obtained, the speechprocessing unit 121 obtains, from the avatar DB 112 and avatar activeinformation 115, the information on all the active avatars existingwithin the effective range across the pseudo wall object ‘i’, theinformation of which is obtained from the avatar ‘a’ in S61 (S63). Thisobtaining is performed in order to store the speech in the speech DB111, in association with addressed avatars. The avatar activeinformation 115 will be described later in the description of thefiltering unit 123.

The speech processing unit 121 determines whether or not the informationon the avatars is obtained in S63 (S64). If the information is obtained,then the filtering unit 123 starts processing, and updates the speech DB111 after processing the speech through the filter (S65). The filteringunit 123 will be described in more detail later. If there are two ormore pseudo wall objects, the steps S61 to S65 are repeatedly executed anumber of times as many as the number of the pseudo wall objects, eitherafter it turns out that the information on any avatar is not obtained inS64, or after the filtering unit 123 completes the processing in S65.After that, the speech processing unit 121 obtains the information on anavatar existing within the reach of the speech on the same side of thepseudo wall object ‘i’ as the avatar ‘a’ (S66). If the avatarinformation is obtained (S67), the speech processing unit 121 stores thespeech information in the speech DB 111 in association with theaddressed avatar (S68).

Chat Filtering

The pseudo wall object can have a chat filter function. The pseudo wallobject having the chat filter function converts a text document inputtedby an avatar, and thus another avatar can read the text document thusconverted. In addition, an avatar can also read a text document in thevirtual world after the pseudo wall object having the chat filterfunction converts the text document. As described above, the chatprocessing unit 122 in the server performs the processing for chat.

FIG. 7 shows a filter processing flow in the chat processing unit. Here,the processing for chat between avatars is explained. At first, when theuser of the avatar ‘a’ inputs a text document from the input unit 227such as a keyboard, the text document is inputted and stored into thechat DB 113 through the information update units 226 and 126 of theclient terminal and the server, respectively (S70). Next, the chatprocessing unit 122 obtains, from the pseudo wall object DB 110, theinformation on the pseudo wall object existing within the reach of thechat from the avatar ‘a’ (S71). A range of the reach of the chat can beset by the user of the avatar ‘a’, or can be automatically set bydistance. The chat processing unit 122 determines whether or not theinformation on the pseudo wall object is obtained in S71 (S72), and thenobtains, from the avatar DB 112 and the avatar active information 115,the information on all the active avatars existing within the effectiverange across the pseudo wall object ‘i’, the information of which isobtained from the avatar ‘a’ in S72 (S73). Here, the avatar activeinformation 115 is temporarily memorized information. The avatar activeinformation 115 will be described later in the description of thefiltering unit 123.

The chat processing unit 122 determines whether the information on theavatars is obtained in S73 (S74). If the information is obtained, thenthe filtering unit 123 starts processing, and updates the chat DB 113after processing the text document through the filter (S75). Thefiltering unit 123 will be described in more detail later. If there aretwo or more pseudo wall objects, the steps S71 to S75 are repeatedlyexecuted a number of times as many as the number of the pseudo wallobjects, either after it turns out that the information on any avatar isnot obtained in S74 or after the filtering unit completes the processingin S75. After that, the chat processing unit 122 obtains the informationon an avatar existing within the reach of the chat on the same side ofthe pseudo wall object as the avatar (S76). If the avatar information isobtained (S77), the chat processing unit 122 stores the chat informationin the chat DB 113 in association with the addressed avatar (S78).

Hereinafter, the filter processing in the speech processing unit and thechat processing unit will be described. FIG. 8 shows a processing flowof the filtering unit 123. The filtering unit 123 determines whether ornot the update information is inputted (S80). If the update informationis inputted, the filtering unit 123 performs the filter processing. Whenthe update information is inputted, the filtering unit 123 determineswhether the update information is for chat (S81). If the updateinformation is not for chat, the filtering unit 123 filters the updateinformation as that for speech. Specifically, the filtering unit 123obtains the data from the speech DB 111 (S82 a), and determines whethera direction of the update information relative to the pseudo wall objectis the forward direction (S83 a). If it is the forward direction, thefiltering unit 123 performs the speech filtering (forward direction)defined in the pseudo wall object DB 110 (S84 a). In contrast, if thedirection of the update information is the reverse direction, thefiltering unit 123 performs the speech filtering (reverse direction)defined in the pseudo wall object DB 110 (S85 a). After the execution ofthe filtering, the filtering unit 123 updates the information in thespeech DB 111 to the information obtained through the speech filter (S86a).

If the update information is for chat, the filtering unit 123 performsprocessing similar to the case of processing the update information forspeech. Specifically, the filtering unit 123 obtains the data from thechat DB 113 (S82 b), and determines whether a direction of the updateinformation relative to the pseudo wall object is the forward direction(S83 b). If it is the forward direction, the filtering unit 123 performsthe chat filtering (forward direction) defined in the pseudo wall objectDB 110 (S84 b). In contrast, if the direction of the update informationis the reverse direction, the filtering unit 123 performs the chatfiltering (reverse direction) defined in the pseudo wall object DB 110(S85 b). After the execution of the filtering, the filtering unit 123updates the information in the speech DB 111 to the information obtainedthrough the speech filter (S86 b). If after S80 there is no updateinformation or if after the chat DB 113 or the speech DB 111 is updated,there is update information, the filtering unit 123 determines whetheror not an avatar existing within the reach of speech or chat is inactive(S87). If the avatar is inactive, the filtering unit 123 updates theavatar active information 115 (S88). When an avatar is inactive, avirtual space component does not reach the inactive avatar. Thefiltering unit 123 excludes such an inactive avatar from the target forthe speech processing by storing the information on the inactive avatarin the avatar active information 115. The avatar active information 115is temporarily memorized in a main memory or the like.

Similarly, the pseudo wall object can also have a light filteringfunction. The light filtering function is for changing the appearancesof the brightness and the 3D model of an object across the pseudo wallobject. The processing flow is the same as the foregoing flows forspeech and chat except for that a filter processing unit is included inthe light processing unit. By taking the speech processing unit as anexample, the processing flow of the light filtering function can beshown by replacing the speech DB 111 with the avatar DB 212 and the 3Dmodel DB 214.

In addition, the pseudo wall object can also have an object filterfunction. When a pseudo wall object allows an object to passtherethrough, the object filter function is able to make a change in anobject after the object passes therethrough. This function isimplemented by changing the solid attribute of a pseudo wall object. Theprocessing flow of the object processing unit is similar to that of thelight processing unit. Additionally, the object processing unit performsthe process of determining whether or not to allow an object to passthrough the pseudo wall object according to the solid attribute of thepseudo wall object. This processing is peculiar to the object processingunit, and accordingly will be specially explained below.

Determination as to Passage of Object

The pseudo wall object is made of a solid attribute. The solid attributedetermines whether or not the pseudo wall object allows avatars and thelike to pass therethrough. If the pseudo wall object is non-solid, otherobjects can move without being influenced by the pseudo wall object. Onthe other hand, if the pseudo wall object is solid, the pseudo wallobject does not allow other objects to pass therethrough, thus makingthe boundary of a space inside the virtual world. Moreover, the solidattribute can be defined in relation to a moving direction. Hence, thepseudo wall objects can form a space that allows an object to go out ofthe space to the outside, but not to enter the space from the outside.When its own avatar, a related object or the like attempts to passthrough and move across a pseudo wall object in the forward direction,the object processing unit 224 determines whether or not the concernedobject can thus move, by referring to the object filter (forwarddirection) in the pseudo wall object DB 110. The object filter (reversedirection) is referred to for an object attempting to pass in thereverse direction. If it is determined that the object can move, thelight processing unit 225 carries out the processing for rendering animage after the movement.

FIG. 9 shows a processing flow for movement determination in the objectprocessing unit 224. Upon receipt of a request to move the own avatar, arelated object or the like, the object processing unit 224 obtains theinformation from the avatar DB 212 and performs calculation to obtain anew 3D location after the movement (S90). Thereafter, the objectprocessing unit 224 obtains the information on objects existing withinthe movement distance from the avatar DB 212, the pseudo wall object DB210 and the 3D model DB 214 (S91). The information on the objectsexisting within the movement distance is sequentially obtained inascending order in terms of the distance from the own avatar or therelated object. The object processing unit 224 determines whether or notthe object information is obtained in S91 (S92). If the objectinformation is obtained, the object processing unit 224 determineswhether or not each of the objects is a non-solid pseudo wall object byreferring to the object filter according to the current moving directionin the pseudo wall object DB 210 (S93). If all the pseudo wall objectsare non-solid, steps S91 to S93 are repeatedly executed a number oftimes as many as the number of the objects existing within the movementdistance. This is because the avatar can pass through all the objects ifthe objects are non-solid. Upon finding a solid pseudo wall objectwithin the movement distance, the object processing unit 224 thendetermines that the avatar cannot move (S94) and terminates theprocessing. In contrast, if no solid pseudo wall object is found, theobject processing unit 224 then determines that the avatar can move(S95) and terminates the processing. According to the abovedetermination, the light processing unit 225 performs the processing forrendering and updates the information in the DBs in the client terminaland the DBs in the server.

Transparency of Pseudo Wall Object

The pseudo wall object can be set to be semi-transparent, to be fulltransparent though having a shape like a glass, or to be completelyinvisible. In this case, scenery and the like on the other side of thepseudo wall object are rendered on the screen. On the other hand, thepseudo wall object can be set to be opaque. In this case, the other sideof the wall is not rendered on the screen. The light processing unit 225(i) determines whether or not the wall is transparent and also whetheror not to render the other side of the wall according to thedetermination on wall transparency (ii) performs the processingaccompanying such determinations and necessary for rendering.

FIG. 10 shows a rendering processing flow in the light processing unit225. The light processing unit 225 obtains the information on objectsand scenery within a field of vision (called in-sight object-sceneryinformation) from the pseudo wall object DB 210, the avatar DB 212 andthe 3D model DB 214 (S100). The in-sight object-scenery information isobtained in ascending order in terms of the distance.

The light processing unit 225 determines whether or not the in-sightobject-scenery information is obtained in S100 (S101). Then, the lightprocessing unit 225 determines whether a new object two-dimensionallymapped from the object/scenery information obtained in S100 completelyoverlaps with any existing object stored in a 2D rendered area 215 (S102and S103). This is because if a certain object exists in the front, apseudo wall object or other objects behind the certain object arecompletely covered with the certain object when being renderedtwo-dimensionally, and thus do not need to be rendered. If the twoobjects do not completely overlap with each other, a 2D rendered section(S107) renders an image for a part of the new object not overlappingwith the existing object (S104). The 2D rendered section is included inthe light processing unit and performs processing for converting athree-dimensional image into a two-dimensional image.

If the object is not a semi-transparent or transparent pseudo wallobject (S105), the light processing unit 225 adds an obtained object tooverlapping areas (S106). This is because an object that is notsemi-transparent or transparent obstructs the view behind the object.The overlapping areas are stored in the 2D rendered area 215 and used todetermine the overlapping. The 2D rendered area 215 temporarily storesobjects other than semi-transparent and transparent pseudo wall objectsif the objects do not each overlap with any other object or each existin front while overlapping with another object. The 2D rendered area 215is a temporary memory and is memorized in a main memory or the like.Steps S100 to S106 are repeatedly executed a number of times, as many asthe number of objects within the field of vision, after it turns outthat the two objects completely overlap with each other, after S105 ifthe object is a semi-transparent or transparent wall, or after S106 ifthe object is not a semi-transparent or transparent wall. The processingis terminated when there are no more objects to be obtained within thefield of vision.

Hereinbelow, descriptions will be provided for examples of the pseudowall object according to an embodiment of the present invention.

Example 1

Automatic Translation Pseudo Wall Object

One specific example of the speech filtering of the pseudo wall objectis an automatic translation pseudo wall object. FIG. 11 shows a screenimage when the automatic translation pseudo wall object is used.

Consider the situation where an avatar 2 who is a Japanese speaker andan avatar 3 who is an English speaker talk to each other. In thissituation, a speech 4 in English made by the avatar 3 of the Englishspeaker is converted into a speech 5 in Japanese by an automatictranslation pseudo wall object 1 and is listened to by the avatar 2 ofthe Japanese speaker. Conversely, a speech 6 in Japanese made by theavatar 2 of the Japanese speaker is converted into a speech 7 in Englishby the automatic translation pseudo wall object 1 and is listened to bythe avatar 3 of the English speaker. In this case, the pseudo wallobject DB 110 stores the definitions of: a direction from 3 to 2 as theforward direction (normal); a function of translating English intoJapanese as the speech filter (forward direction); and a function oftranslating Japanese into English as the speech filter (reversedirection).

With these definitions, the automatic translation pseudo wall object 1plays a role called an interpreter in the real world, and enablesavatars speaking in different languages to have smooth conversions. Theautomatic translation pseudo wall object is also one specific example ofthe chat filtering.

Example 2

Time Operation Pseudo Wall Object

A time operation pseudo wall object is explained as another specificexample of the pseudo wall object having the speech filtering function.Unlike the case of chat, when two or more avatars have a voiceconversation, there is a situation where a speech of an avatar overlapswith a speech of another avatar. In this case, an avatar as a listenerfails to catch each of the two speeches since the two speeches are mixedtogether. This problem can be solved by creating a time operation pseudowall object. When two or more avatars make speech at the same time, thetime operation pseudo wall object provides a time difference betweenspeech timings by prioritizing speeches and thereby enables a listeningavatar to listen to the speeches separately. In other words, the timeoperation pseudo wall object allows individual speeches of avatars to beclearly heard by preventing the speeches from being heard in anoverlapping manner.

Example 3

Avatar Selection Pseudo Wall Object

An avatar selection pseudo wall object is explained as another specificexample of the pseudo wall object having the speech filtering function.FIG. 12 shows a screen image when the avatar selection pseudo wallobject is used. Table 1 shows whether a speech from or to an avatar 2 ispermitted when the forward direction of the pseudo wall object isdefined as a direction from 2 to 3. Here, Y indicates that a speech ispermitted and N indicates that a speech is not permitted.

TABLE 1 AVATAR 3 AVATAR 3′ AVATAR 3″ FROM AVATAR 2 Y N Y TO AVATAR 2 N YY

In the pseudo wall object DB 210, the contents in the upper row in Table1 are registered in the speech filter (forward direction) and thecontents in the lower row in Table 1 are registered in the speech filter(reverse direction).

Through an avatar selection pseudo wall object 1, a speech 4 of theavatar 2 is transmitted to the permitted avatars 3 and 3″, but is nottransmitted to the unpermitted avatar 3′. Conversely, a speech 6 of theavatar 3 is not permitted and thus not transmitted to the avatar 2,while speeches 6′ and 6″ of the avatars 3′ and 3″ are permitted and thustransmitted to the avatar 2. In other words, when an avatar desires toinform particular avatars of a certain speech while having conversationswith several avatars, the avatar selection pseudo wall object enablesonly the particular avatars to be informed of the speech or disablesonly the other avatar to be informed of the speech. The avatar selectionpseudo wall object is also one specific example of the chat filtering.

Example 4

Speed Change Pseudo Wall Object/Particular Word Block Pseudo Wall Object

In addition to the foregoing specific examples, a pseudo wall object canhave the functions of (i) changing a voice tone, for example, changing amale voice to a female voice; (ii) changing a talking speed, forexample, changing a rapid talk to a slow talk; and (iii) prohibitingonly a particular word from being heard. To prohibit only a particularword from being heard is to make only a part of conversation thatincludes a designated particular word inaudible to a conversationpartner. For example, when one utters “aiueo” while designating “ai” asa particular word, a conversation partner hears “--ueo.”

Example 5

Advertisement Change Pseudo Wall Object

An advertisement change pseudo wall object will be described as aspecific example of the chat filtering. This filtering is fortransmitting a text document between an avatar and an object. FIG. 13shows a screen image when the advertisement change pseudo wall object isused. In the screen, a child avatar 3 and old male avatar 3′ watch anadvertisement 2 for the same product. When the avatars 3 and 3′ watchthe advertisement 2 through the advertisement change pseudo wall object1, the advertisement change pseudo wall object 1 is capable of causingthe avatars 3 and 3′ to view different catch-phrases 4 and 4′respectively suitable to their ages. In this way, in a situation wherean avatar watches an advertisement in a virtual space, the content ofthe advertisement is changed according to characteristics, such as ageand sex, of the avatar by installing an advertisement change pseudo wallobject between the avatar and the advertisement. Besides the foregoingspecific examples, such pseudo wall object is also capable of changingcharacter attributes such as font type and font size and changing fromkanji characters to hiragana characters. By use of a pseudo wall objectprovided with such a chat filer function, as described above, charactersand texts can be changed according to the needs of each of avatar.Accordingly, the avatars are allowed to make smooth communications or toobtain and provide individually useful information.

Example 6

Day and Night Converter Pseudo Wall Object

A day and night converter pseudo wall object is explained as a specificexample of the light filtering. FIG. 14 shows a screen image when apseudo wall object having a function of converting day and night isused. Here, suppose the case where the virtual world is at a daytime andwhere an avatar 2 views an avatar 3 through the day and night converterpseudo wall object 1. In this case, the avatar 2 views the avatar 3 witha dark background as if the avatar 3 is at night. In this way, theactual time of a user of an avatar is expressed.

Example 7

3D Model Change Pseudo Wall Object

A 3D model change pseudo wall object is explained as a specific exampleof the light filtering. For example, when an avatar views an objectthrough the 3D model change pseudo wall object, the 3D model changepseudo wall object allows the avatar to view the object in a magnifiedmanner. Instead, the 3D model change pseudo wall object is also able tocause the avatar to view the object with its texture or the likechanged.

Example 8

Object Selection Change Pseudo Wall Object

An object selection change pseudo wall object is explained as a specificexample of the object filter. Here, a space is created for a meetingroom having a pseudo wall object as its wall. The wall of the meetingroom can be set to allow only authorized avatars to enter the meetingroom during a predetermined time period and to allow every avatar to getout of the meeting room freely after a meeting ends. This function canexclude avatars who should not participate in the meeting and also forceavatars to participate in the meeting until the meeting ends. Moreover,the object selection change pseudo wall object is also capable ofchanging the appearance of each of avatars, who can enter and stay inthe meeting room, in a distinguishable manner according to the divisionand position, and restore the avatar to its original appearance afterthe avatar gets out of the meeting room. Specifically, the objectselection change pseudo wall object is capable of changing the clothesof avatars to cause: an avatar of a user belonging to research anddevelopment to wear a white coat; an avatar of a user belonging to shopfloor to wear a factory uniform; an avatar of a user belonging to asales division to wear a business suit; and the like.

FIG. 15 is a diagram showing a hardware configuration of a server or aclient terminal according to an embodiment of the present invention. Thefollowing provides a description for the general configuration as aninformation processing apparatus typified by a computer. Needless tosay, however, any minimum necessary configuration can be selected for adedicated apparatus or a built-in device according to its useenvironment. Here, the server and the client terminal are collectivelyreferred to as the client terminal/server.

The client terminal/server includes a central processing unit (CPU)1010, a bus line 1005, a communication I/F 1040, a main memory 1050, abasic input output system (BIOS) 1060, a parallel port 1080, a USB port1090, a graphic controller 1020, a VRAM 1024, a speech processor 1030,an I/O controller 1070, and input means such as a key board and mouseadapter 1100 and a microphone 1101. Storage units such as a flexibledisk (FD) drive 1072, a hard disk 1074, an optical disk drive 1076 and asemiconductor memory 1078 can be connected to the I/O controller 1070. Adisplay device 1022 is connected to the graphic controller 1020. Inaddition, an amplifier circuit 1032 and a speaker 1034 are optionallyconnected to the speech processor 1030.

The BIOS 1060 stores a boot program to be executed by the CPU 1010 whenthe client terminal/server is started up, the programs depending onhardware of the server 100, the client terminal 200, and the like. TheFD drive 1072 reads a program or data from a flexible disk 1071 andprovides the read program or data to the main memory 1050 or the harddisk 1074 through the I/O controller 1070.

A DVD-ROM drive, a CD-ROM drive, a DVD-RAM drive and a CD-RAM drive, forexample, are usable as the optical disk drive 1076. In order to use eachof the drives, it is necessary to use an optical disk 1077 in conformitywith the drive. The optical disk drive 1076 also reads a program or datafrom the optical disk 1077 and provides the read program or data to themain memory 1050 or the hard disk 1074 through the I/O controller 1070.

A computer program provided to the client terminal/server is stored in astorage medium such as the flexible disk 1071, the optical disc 1077 ora memory card and thus is provided by a user. Instead, this computerprogram is read from any of the storage media via the I/O controller1070 or downloaded via the communication I/F 1040. Then, the computerprogram is installed on the client terminal/server, and then executed.Operations that the computer program causes the client terminal/serverto execute are the same as the operations in the foregoing clientterminal/server and the description thereof is omitted here.

The foregoing computer program can be stored in an external storagemedium. In addition to the flexible disk 1071, the optical disc 1077 orthe memory card, a magnet-optical storage medium such as a MD and a tapemedium can be used as the storage medium. Alternatively, the computerprogram can be provided to the client terminal/server via acommunication line, by using, as a storage medium, a storage device suchas a hard click or an optical disc library provided in a server systemconnected to a private communication line or the Internet.

The foregoing embodiment mainly explains the client terminal/server, buta computer is also capable of implementing the same functions as thoseof the foregoing client terminal/server. In this case, a program havingthe same functions is installed on the computer. Accordingly, the clientterminal/server described as the embodiment can be achieved by using theforegoing method and a computer program of implementing the method.

The client terminal/server of the present invention can be obtained byemploying hardware, software or a combination of hardware and software,as described above. In a case of implementation using a combination ofhardware and software, a typical example is implementation of a computersystem including a certain program. In this case, the certain program isloaded to the computer system and then executed, thereby the certainprogram causing the computer system to execute processing according tothe present invention. This program is composed of a group ofinstructions each of which can be expressed by use of any language, codeor expression. In accordance with such a group of instructions, thesystem can directly execute specific functions, or can execute thespecific functions after either/both (1) converting the language, codeor expression into another one, or/and (2) copying the instructions intoanother medium. As a matter of course, the scope of the presentinvention also includes not only such a program itself, but also aprogram product including a medium in which such a program is stored. Aprogram for implementing the functions of the present invention can bestored in an arbitrary computer readable medium such as a flexible disk,a MO, a CD-ROM, a DVD, a hard disk device, a ROM, an MRAM and a RAM. Inorder to store the program in a computer readable medium, the programcan be downloaded from another computer system connected to the systemvia a communication line or can be copied from another medium. Moreover,the program can be compressed to be stored in a single storage medium ordivided into several pieces to be stored in several storage media.

According to the present invention, a new method can be provided forcontrolling communications between objects and object movements in avirtual world.

Although the embodiments of the present invention have been describedhereinabove, the present invention is not limited to the foregoingembodiments. Moreover, the effects described in the embodiments of thepresent invention are merely enumerated examples of the most preferableeffects made by the present invention and the effects of the presentinvention are not limited to those described in the embodiments orexamples of the present invention.

We claim:
 1. A method of controlling sharing of information in a virtual world by two or more computing devices connected to each other through a network of interconnected computing devices, the method comprising the steps of: creating a third object between a first object and a second object in the virtual world when said first object and said second object transmit communication information therebetween; and filtering said communication information with said third object according to a direction of transmission of the communication information between the first object and the second object.
 2. The method according to claim 1, further comprising: generating said third object around an object that transmits visual information.
 3. The method according to claim 1, wherein said third object is created as a pseudo wall.
 4. The method according to claim 1, wherein said third object performs different kinds of filtering according to moving directions of information and another object.
 5. The method according to claim 1, wherein said third object has information defining whether or not to allow another object to pass through said third object.
 6. The method according to claim 1, wherein creating includes checking an authority level to determine whether creation of said third object is permitted.
 7. A non-transitory computer readable article of manufacture tangibly embodying computer readable instructions for executing the steps of a method of controlling information shared in a virtual world by a server and a plurality of users' client terminals connected to each other through a network according to claim
 1. 8. A system for controlling, using a processor, sharing of information in a virtual world by two or more computing devices connected to each other through a network of interconnected computing devices, the system comprising: a memory coupled to the processor, the processor being configured to control the sharing of information by: generating a third object between a first object and a second object in the virtual world when said first object and said second object transmit communication information therebetween; storing information on said third object; and said communication information with said third object according to a direction of transmission of the communication information between the first object and the second object.
 9. The system according to claim 8, wherein said third object creating unit creates said third object around an object that transmits visual information.
 10. The system according to claim 8, wherein said third object storage unit holds filtering information for performing different kinds of filtering according to moving directions of information and another object.
 11. The system according to claim 8, wherein said third object storage unit holds information defining whether or not to allow another object to pass through said third object.
 12. The system according to claim 8, wherein said client terminal performs processing for visual information and information on a movement of an object.
 13. The system according to claim 8, wherein said server performs processing for said communication information.
 14. The system according to claim 8, wherein said third object creating unit checks an authority level for creating said third object to determine whether creation of said third object is permitted.
 15. A server for controlling, using a processor, sharing of information in a virtual world by two or more computing devices connected to each other through a network of interconnected computing devices, the server comprising: a memory coupled to the processor, the processor being configured to control the sharing of information by: transmitting and receiving information to and from the two or more computing devices; creating a third object between a first object and a second object when said first object and said second object transmit communication information therebetween; storing information on said third object in the memory; filtering the communication information received from said information update unit according to a direction of transmission of the communication information between the first and the second object; and storing (i) said communication information received from said information update unit and (ii) said communication information filtered by said filtering processing unit in the memory.
 16. The server according to claim 15, wherein said third object creating unit creates said third object around said object that transmits visual information.
 17. The server according to claim 15, wherein said third object is created as a pseudo wall.
 18. The server according to claim 15, wherein said filtering processing unit performs different kinds of filtering according to moving directions of information and another object.
 19. The server according to claim 15, wherein the third object storage unit includes information defining whether or not to allow another object to pass through said third object.
 20. The server according to claim 15, further comprising: a visual information storage unit; a movement information storage unit; and a communication information storage unit; wherein said visual information storage unit, said movement information storage unit and said communication information storage unit respectively stores, through said information update unit, visual information, information on movements of objects, and said communication information received from said client terminals. 